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Abstract 



Distributed power control over interference limited network has received an increasing intensity of interest over 

the past few years. Distributed solutions (like the iterative water-filling, gradient projection, etc.) have been intensively 

ly-j ■ investigated under quasi-static channels. However, as such distributed solutions involve iterative updating and explicit 

message passing, it is unrealistic to assume that the wireless channel remains unchanged during the iterations. 

Unfortunately, the behavior of those distributed solutions under time-varying channels is in general unknown. In this 

paper, we shall investigate the distributed scaled gradient projection algorithm (DSGPA) in a K pairs multicarrier 

interference network under a finite-state Markov channel (FSMC) model. We shall analyze the convergence property 

c/5 \ as well as tracking performance of the proposed DSGPA. Our analysis shows that the proposed DSGPA converges 

O . to a limit region rather than a single point under the FSMC model. We also show that the order of growth of the 

tracking errors is given by O (l/N), where N is the average sojourn time of the FSMC. Based on the analysis, 

we shall derive the tracking error optimal scaling matrices via Markov decision process modeling. We shall show 

that the tracking error optimal scaling matrices can be implemented distributively at each transmitter. The numerical 

results show the superior performance of the proposed DSGPA over three baseline schemes, such as the gradient 

projection algorithm with a constant stepsize. 



Index Terms 

Multicarrier Interference Network, Distributed Power Control, Time-varying Channel, Region Stability, Tracking 
Error Analysis, Tracking Error Optimization 

I. Introduction 

Power control algorithm design over interference limited network has received an increasing intensity of interest 
over the past few years. In UJ, the power control design over K pairs interference network is formulated as a 
deterministic non-cooperative game and distributive solution, namely the iterative water-filling algorithm is proposed 
to achieve the Nash Equilibrium (NE) of the game. The distributed power control algorithm design in interference 
network has also been studied in O, using game theory, and from the distributed network utility maximization 
(NUM) point of view |4), Q. There are also some other works on distributed power allocation in interference 
networks, such as the asynchronous distributed pricing (ADP) framework J5J, Q and the adjoint network 1 based 
approach (H, & In all these works, distributive solutions are critical in networks where many transmitter-receiver 
pairs are randomly placed. In the absence of an infrastructure linking all these nodes, centralized solutions are 
difficult to implement due to the difficulty of gathering global knowledge of channel state information (CSI) as well 
as non-scalable limitations in terms of complexity. The implementation of such distributive algorithms, such as the 
distributed gradient projection algorithm (3J, the iterative water-filling algorithm fT], and the gossip algorithm |[T0l , 
often involves iterative solution with explicit message passing. Moreover, in all these existing works, the convergence 
and the optimality properties of the algorithms are established under the quasi-static channel assumption. Specifically, 

In the adjoint network approach, the authors j§J, (5) first derive a computational algorithm to obtain a global optimal solution. The 
computational algorithm requires global observations across the wireless networks. The authors (8), (9) then proposed an efficient mechanism 
based on adjoint network to distribute the global observations across the nodes. 



the CSI is assumed to be quasi-static throughout the iteration process. However, since there are explicit message 
passing in between each iterative step, it is quite unrealistic to assume that the channel remains unchanged over 
a significant number of iterative steps for the algorithm to converge. As a result, it is of great importance both 
theoretically and practically to investigate the behavior of the distributive algorithms ifTI- lfTOl under time-varying 
channels. In this paper, we are interested in a network topology consisting of K transmit-receive pairs sharing a 
common spectrum with Np independent subbands. For a distributive power control algorithm in such interference 
networks over time-varying channels, the NE, which is a function of the CSI, will be time-varying as well. Hence, 
the following are some important questions, which still require more investigations. 

• How fast could the wireless channel change before the distributive algorithm failed to track the moving NE? 

• Given that the distributed algorithms can track the moving NE of the interference network, can we obtain 
closed-form bounds on the tracking errors? 

• How to enhance the existing distributive algorithm (designed for quasi-static CSI) to optimize the associated 
tracking performance! 

Due to the randomness of the wireless channel and the nonlinear dynamics of the iteration process encountered, 
it is highly nontrivial to answer the above questions in general. There are some preliminary works on distributed 
power control in time-varying interference networks. For instance, in ifTTTl . the hybrid system model was used to 
study the multicell CDMA interference game. While the existing works ifTTI provide some preliminary results on the 
behavior of the distributed power control algorithm under time-varying channels, the techniques cannot be utilized 
in the vector interference game that we are considering. For instance, the authors of flTTl established the region 
stability 2 of the gradient play of a multicell CDMA interference game. However, their results cannot be utilized in 
our case because they did not consider the transmit power constraint. Furthermore, to our best knowledge, none of 
the existing works have investigated the closed-form tracking error expressions as well as enhancing the distributive 
algorithm to optimize the associated tracking performance. On the other hand, parameter tracking based on the 
linear regression model and least mean square (LMS) algorithm in nonstationary environments was investigated 
in |[T3l , Ifl4ll . Also, the authors of |fT31l , lfT6l studied the problem of tracking the optima of discrete stochastic 
optimization in time-varying scenarios. However, the techniques and results of lfT3l - |[T6l . which are based on the 
special structure of the underlying dynamics, cannot be applied to the distributed gradient projection algorithm we 
are considering. 

In this paper, we shall shed some lights on the above open questions. We shall model the transient of the 
distributive power control algorithm in the A' -pair interference network as an algorithm trajectory of a nonlinear 
system ifTTll . Based on randomly switched system modeling, we first establish the region stability and the technical 
conditions for the convergence of the distributive power control algorithm under a finite-state Markov channel 
(FSMC) model. Based on that, we shall derive closed-form order of growth of the tracking errors, namely the 
expected-absolute-error (EAE) as well as the mean-square-error (MSE), between the algorithm trajectory and the 
moving NE. Based on these results, we shall enhance the tracking performance of the distributive power control 
algorithm using a novel Scaling Matrix Optimization. Specifically, we shall determine the closed-form optimal 
scaling matrices in the iterative update of the power control algorithm so as to minimize a general function of the 
tracking errors. The optimal scaling matrix is adaptive to the current CSI and can be computed distributively at 
each transmitter based on local CSI only. As a result, the solution could be implemented with low complexity. 

The paper is organized as follows. In Section [TTJ we introduce the interference network model, FSMC model 
as well as the formulation of the power control game. In Section |III1 we elaborate the distributed scaled gradient 
projection algorithm and analyze its convergence behavior as well as tracking errors via switched system modeling. 
In Section [IV] we shall first introduce a dominating error process and we shall then formulate the tracking error 
minimization problem via MDP modeling and derive the optimal solution of the scaling matrices. Section [V] 
demonstrates the tracking performance of the proposed algorithm and verifies the analytical results by simulations. 
Finally, we conclude with a summary of the main results in Section [V]] 

Notations: Matrix and vectors are denoted with capitalized and small boldface letters, respectively. A T and A* 
denote the transpose and complex conjugate of matrix A, respectively. \ max (A) and X m in (A) denote the largest 
eigenvalue and smallest eigenvalue of matrix A, respectively. [A] lm denotes the (l,m) th entry of matrix A, and 

2 A nonlinear dynamic system is said to be region stable if the trajectory of the system converges globally asymptotically to a limit region 
G2) 



In f denotes the Np x Np identity matrix. C, R and R + denote the set of complex numbers, real numbers and 
non-negative real numbers, respectively. E denotes the operation of taking expectation, (g) denotes the operation 
of Cartesian product, and Pr {tp} denotes the probability of set ijj. Finally, 1 denotes a column vector of all ones 
with appropriate dimension. 

II. System Model 

In this section, we shall introduce the time-varying i\T-pair multicarrier interference network model as well as 
the formulation of the deterministic non-cooperative power control game. 

A. K-pair Multicarrier Interference Network Model 

Consider a time-varying multicarrier interference network with K transmitter-receiver pairs sharing Np non- 
overlapping subcarriers, as shown in Fig. Q] Denoting /C and S as the set of transmitters and subcarriers respectively, 
the baseband signal model at the n th time-slot can be written as 

K 

i=i 
where 

- yi (n) denotes the received signal at the k th receiver on the s th subcarrier; 

- n£: (n) denotes the channel coefficient between the k th receiver and the j th transmitter on the s th subcarrier; 

- p- s (n) denotes the transmit power of the j th transmitter on the s th subcarrier; 

- af (n) denotes the transmitted symbol of the j th transmitter on the s th subcarrier, with normalized power, 
i.e., E{4' ) (n)4* ) *(n)}=l; 

- zj? (n) denotes the additive white Gaussian noise at the k th receiver on the s th subcarrier, the power of which 

is given by E < z£ (n)z£ *{n) > = a 2 . 

Based on the signal model given in equation (Q]), the instantaneous mutual information (in nats per channel use) 
of the k th link at time-slot n can be written as 

N F 

C k (n) = J2 log (l + 7i S V)) , V k e JC, (2) 

8=1 

where % (n) denotes the receiving signal to interference plus noise ratio (SINR) at the k th receiver on the s th 
subcarrier, which is given by 



»^ _ gi S k( n )pi s \ n ) 



Ti SJ («) = - Jr' X :\ (,),. . V k € JC,V s E S, (3) 



where g^.-{n) = h^Hn) is the power gain of the fading channel coefficient h^Un). 
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B. Finite-State Markov Channel Model 

Motivated by the accuracy and simplicity of the FSMC model lfT8l - |[23l for time-varying channels 3 , we model 
the channel process < nj*:(n) > as an ergodic finite-state Markov chain, V k, j € JC, s G S. Let the state space of 

the FSMC {4?( f 0} be Mkj> with cardinality Cj[ s ] , V k,j G K,s G S. For the FSMC {4? (")}' we make the 
following assumptions. 

3 The current model of time-varying finite state Markov Fading Channel (FSMC) is a very commonly accepted model which has been 
widely used in a lot of literature such as |18|-|23| to model the time-varying fading channels. This model is not too complicated so that it 
is analytically tractable and it is complicated enough to give us some first order insights. The results in this paper can be extended to the 
case with continuous state space but the extension to continuous state space Markov fading channel involves some mathematical technicality. 
The extension can be considered as the limit of a sequence of finite state Markov Chain (FSMC) models 1241 . 1251 . 



l(») 



Assumption 1 (Assumptions on the FSMC): Similar to 11221 . ||23l , the transition probability matrix TL E 
of the FSMC < h k -(n) > is assumed to have the following structure: 
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where v = 1 — 2e and e = {fdj), with /^ and r denoting the doppler frequency shift and symbol duration, 
respectively. ■ 

Let h(n) G C^ Arj?xl denote the collection of all the fading channel coefficients 4 , i.e., 



h(n) = {h$(n), V (£;,j,s)E/C®/C®s} G 



^k 2 n f xi 



(5) 



then {h(n)} is also an ergodic finite-state Markov chain [20]. The state space H. and transition probability matrix 
T of the FSMC {h(n)} are given by (H 



H 



«« and T 
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oQxQ 



(6) 



{{k,j,s)elC®K®S} 



{(fcj»e/c®/C(g)5} 



where Q = |H| = ri{(fe j, s )ex:®x;®5} Qfcj * s tne cardinality of the state space H = {hi, h 2 , ■ • • , Iiq} of the FMSC 
{h(n)}. For notational convenience, we use q G Q = {1, 2, • • • , Q} as an index to enumerate the state realization 
of the CSI {h(n) G W}. 



C Power Control Game in the K-pair Interference Network 

In the strategic noncooperative game formulation of the distributed power control problem in quasi-static interfer- 
ence networks, the players are the K active links and the payoff functions are the instantaneous mutual information 
of the active links 0, Gl. As a result, for each realization of the CSI {h(n)}, the power control game formulation 5 
is summarized below. 

Problem I (Multicarrier Interference Game): At each time-slot n, given the transmit power profile p„^(n) of 
the other players, the k th player tries to maximizes its own payoff function via solving the following capacity 
maximization problem: 



maximize C k {p k (n),p_ k (n)) w; „ 
subject to PfcW G iZ fc 



(7) 



where 



- Cfc(pfc(n), p_fc(n)) = Cfc(n) is the instantaneous capacity of the /c t/l link given in (0; 

- Pfe(n) denotes the transmit power profile of the /c t/l player, i.e., 

Pfc(n) = Pfe (n) p[ (n) ■■■ p[ F '(n) ,V A; G /C; 

- p_fc(n) denotes the transmit power profile of all players excluding the /c tft player, i.e., 

p-fc(n) = [pf (n) pl'(n) ••• Pk_i(n) pf+i («)••• Pk( u )] ,VfceJC; 

- the strategy set fij. of the /c tft player is given by 



n* 



p/, 



Pfc G 



^, i T Pfc < p fc , maa; , Pfe b o !> , v k g /c, 



(8) 



(9) 



(10) 



Here, we assume independent sub-channels for simplicity. However, all the results stated in this paper hold for correlated sub-channels 
as well. 

The non-cooperative game considered in this paper serves more like a motivating example to study the issue of convergence of iterative 
algorithms under time-varying CSI. The approach considered in this paper can also be applied to other contraction-based iterative algorithms 
as well (26). 



with Pk,max denoting the maximum power budget at the k th transmitter. 

One widely adopted optimality criterion for the game formulation given in ([7]) is the achievement of a Nash 
Equilibrium (NE) 0, which is formally defined below. 

Definition 1: A set of (pure) strategies is a Nash Equilibrium of Game Q if no player can benefit by unilaterally 
changing its strategy. Mathematically, a (pure) strategy profile p*(n) = (p£(n),pl fe (n)) is a NE of Game Q if 

Ck (Pfc(n),pl fc (n)) > C k (p fc (n),pi fc (n)) , V p fc (n) G U k , V fc G /C. (11) 

In quasi-static interference networks, the iterative water-filling algorithm (IWFA) as well as the gradient projection 
algorithm (GPA) were proposed to solve Game Q Q-Gl. The convergence of these algorithms to the static NE were 
established based on the contraction mapping theory IJ271 , i.e., the IWFA and GPA were shown to be contraction 
mapping (TJ-lEl. Moreover, for each update of the power allocation vector p k (n), the k th transmitter requires 
the feedback of local interference level at the k th receiver. As a result, one cannot assume that the CSI remains 
unchanged for many iterative updates. However, the convergence behavior of the IWFA and GPA under time-varying 
channels is in general unknown. 

Throughout this paper, we assume that the channel coefficients < hj? k , V s € S > are known perfectly at both ends 

of the k th link, and the k th receiver measures local total received power {<r 2 + Ylj=i9kj ?( n )Pj ( n )> V s 6 5} 
and feeds back this information to the k th transmitter at the end of each time-slot, V k G /C. Table U summarizes 
the main notations used in the paper. 

III. Switched System Modeling and Convergence Behavior Analysis 

In general, the transient behavior of an iterative algorithm can be characterized by an algorithm trajectory of 
an associated nonlinear dynamic system and the NE is the associated equilibrium point of the nonlinear dynamic 
system. Since the NE is a function of the CSI, a quasi-static CSI corresponds to a nonlinear system with static 
equilibrium point. The convergence behavior of the iterative algorithm can be visualized as the algorithm trajectory 
converging to the equilibrium point as illustrated in Fig. [2] On the other hand, time-varying random CSI corresponds 
to a randomly moving NE (or a randomly moving equilibrium point) and the convergence behavior of the algorithm 
can be visualized as how well the algorithm trajectory could track the moving equilibrium point of the nonlinear 
system. In this section, we shall first propose a novel distributive scaled gradient power control algorithm for the 
interference Game Q under time-varying CSI. Based on this, we shall utilize the randomly switched system nonlinear 
control theory to analyze the convergence behavior as well as the tracking errors of the proposed scaled gradient 
projection algorithm. 

A. Distributed Scaled Gradient Projection Algorithm 

The existence and uniqueness of NE of Game Q under quasi-static channels has been extensively studied (e.g., 
see HI] — (3J and references therein.). The solution set of Game Q is nonempty under any channel conditions, while 
the uniqueness of NE depends on the channel coefficients of the whole network (TJ, ||2). As a result, with the time- 
varying channels, the NE of Game Q changes with time. We thus propose a distributed scaled gradient projection 
algorithm (DSGPA) to track the moving NE of Game Q in the time-varying interference networks. 

Different from the gradient projection algorithms proposed in 0, ll28l under quasi-static channels, where the 
scaling matrices are constant, the scaling matrices in our proposed DSGPA can be adaptive to the CSI of the current 
update. Moreover, the scaling matrices could be optimized to minimize the tracking error, which will be detailed in 
section [TV] Before summarizing the main algorithm, we first form the Lagrangian ||29l of the optimization problem 
©: 

A (Pfe(n, A fe (rt)) = C k (p k (n),p_ k (n)) + \ k (n) (P k , m ax ~ l T Pfc(») ,Vfc G /C, (12) 

where X k (n) > is Lagrangian multiplier associated with the sum-power constraint at the k th transmitter. As has 
been established in ll30l . solving the concave programming defined in equation (0 and finding the saddle point 6 

6 A point (Pfc(n), Afe(n)) is a saddle point of the Lagrangian Ck (pfe(n), Afc(n)) if 

£fc(pfc(n),A£(n)) < £ fc (Pk(n),Xt{n)) < C k (p!b(n), Ajt(n)) . (13) 



of the Lagrangian Ck (pfc(n), Afc(n)) is equivalent. We thus adopt the primal-dual gradient method ||30l to find the 
saddle point of the Lagrangian Ck (Pfc(ra, Ajt(n)). The proposed DSGPA is summarized in Algorithm^ 

Algorithm 1 Distributed Scaled Gradient Projection Algorithm (DSGPA) 

• Initialization: 

Set pfc(O) to be any feasible power allocation vector; Afc(O) to be any positive number; and the scaling matrix 
Dfc(O) to be I Np , VfcG/C; 

• Updating the Lagrangian Multiplier Afc(n): 

At each time-slot n (n > 1), after receiving the received power profile pk(n) (see equation (fT6l ) below) fed 
back by the k th receiver, the k th transmitter updates its Lagrangian Multiplier Afc(n) S M+ according to 

A fc (n+ 1) = [Afc(n) - a (P fc , maa; - l T p fc (n))] + ; (14) 

• Updating the power allocation vector Pfc(n): 

The k th transmitter updates its power allocation vector pfc(n) € M + F according to 

p fc (n + l)= [p fc (n)+D- 1 (?i)f fc (?i)] + . (15) 

In Algorithm [7J 

- the k th transmitter needs the channel gain I h^,\fs £5> feedback from its intended receiver k; 

- the received power profile pk{n) is given by 

T 



C 1 )/^ J 2 )^ J.***), 



Pk{n)= pl>{n) p k '{n) ■■■ p k F) (n) 



ykefC, (16) 



with pj. in) = a 2 + J2j = ig^(n)p s (n) denoting the total received power on the s th subcarrier at the k th 
receiver, V s G S, k G K,; 

- a is a stepsize, which can be computed efficiently via the exact line search method ||29l , as the Lagrangian 
given by (fT2l) is linear in Afc(n); 

- D^ (n) denotes the symmetric positive definite scaling matrix, defined in equation (|4TI ); to facilitate distributive 
computation of the dynamic scaling matrices at each node, we impose a block diagonal structure 7 on the scaling 
matrix D^ x (n); 

- ffc(ra) is defined as the gradient of Ck (pk(n, Afc(n)) w.r.t. pfc(n) evaluated at p(n), i.e., 

" (1)/ \ (2)/ n (N F ), s-< T 

9kk( n ) 9 k k(n) g y kk \n) 



h(n) 



f>i 1] (n) p{ 2 \n) p^\n) 



A fc (n + l)l; (17) 



- [a] + = max{a, 0}, which shall be understood componentwisely. 

Remark 1: In Algorithm^ at each time slot n, the k th transmitter updates its power allocation vector based on 
the local information I g^ ;(n),V s G K, >, Pfe(n), and the local receive power profile pfc(n) of the k th receiver. 
Therefore, algorithm\l}can be implemented distributively, without requiring any global information of the network. 

Remark 2: As the FSMC {h(?i)} jumps from one state to another randomly, the NE of Game Q also changes 
with time randomly. As a result, Algorithm [7] would not converge to a single point but rather designed to track the 
moving NE of Game Q. The tracking performance of Algorithm [7J is the focus of the rest of this paper. 

B. Randomly Switched System Modeling 

Randomly switched systems are piecewise deterministic stochastic systems, i.e., between any two consecutive 
switching instants, the dynamics are deterministic lilTl . Formally, a discrete-time randomly switched system is 
defined as follows ifTTl . 

We shall illustrate in Section [V] that there is only very small performance penalty associated with the block diagonal scaling matrix. 



Definition 2 (Discrete-time Randomly Switched System): A Discrete-time Randomly switched system consists of 
a family of subsystems, and a random switching signal that specifies the active subsystem at every time-slot. 
Mathematically, 

x(rt-r-l) = J^(x(n)),when t (n) = u G U = {1,2, ■ ■ ■ ,£/}, (18) 

where x(n) denotes the system state; & u (x(n)) denotes the u th subsystem; and r (n) is the switching signal with 
state space U. ■ 

For the FSMC {h(ra)}, the channel process stays in a state h q for a random sojourn time of N q time-slots, and then 
jumps to another state randomly. During the N q time-slots, the channel coefficients {h(n)} remain constant (i.e., 
h(n) = h q ) and the system is deterministic. We thus can model the time-varying interference network embedded 
with the dynamics of the proposed DSGPA as a randomly switched system, with the FSMC {h(n)} being the 
switching signal and the channel state h q corresponding to the q th subsystem ifTTl . for all q G Q. 

To obtain the dynamics of the randomly switched system model of the power control game explicitly, we rewrite 
the K block-component iterations given by equation (031 ) in Algorithm [7] into one vector form: 

p(n + l) = [p(n) + D- 1 (n)f(?i)] + ^^(p(n)),whenh(n) = h q G H, (19) 

where 

- p(ra) denotes the transmit power profile of all the players, i.e., 

p(n)=[pf(n) p T 2 (n) ■■■ p£(n)] T ; (20) 

- D _1 (n) = blkdlg {Dj -1 (n) ,D^ (n) , • • • ,D^(n)} G £KN F xKN F denotes the block-diagonal scaling matrix 
consisting of the K scaling matrices |D^" 1 (n), V k G /C}; 

- f(n) G 1Z KNf denotes the collection of the K gradient functions {ffc(n), V k G /C}, i.e., 

f(n)=[f 1 T (n) f 2 T (re) ••• f£(n)] T ; (21) 

- S? q (p(n)) denotes the dynamics of the q th subsystem, V q G Q. 

Remark 3: In the above randomly switched system modeling, p(n) is the system state vector, and the Q states of 
the FMSC {h(n)} correspond to the Q subsystems |5i(p(n)), ^(pfn)), • • • , 5g(p(n))}. As the FMSC {h(n)} 
jumps between different states, the switched system ( fT9l switches between different subsystems. 

To simplify the analysis in the sequel, we make the following assumptions. 

Assumption 2 (Existence of NE): We assume that the interference Game ^? in (O has a unique NE for all q G Q. 
In other words, using the sufficient conditions for the existence and uniqueness of NE of Game Q given in 12, 
121 , we assume that 

9kj \ n )Pj,max 1 . . 

max-T^j < 17 -,VK,je£,Kf j. (22) 

^fcfc v^)^k,max 

Remark 4: Under Assumption the proposed distributed scaled gradient projection algorithm (DSGPA) con- 



verges linearly for each channel state 
is static (cardinality of the CSI space 



2]. In other words, the proposed DSGPA converges linearly if the channel 
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Qil = D- 



C. Convergence Analysis of the Proposed DSGPA 

The region stability is a widely used performance measure of iterative algorithms in time-varying environments, 
especially for switched and hybrid systems IfTTl . Ifl2l . When the CSI is time- varying (e.g., the FSMC model), the 
equilibrium point of the system (e.g., the NE) is also changing and hence, the algorithm trajectory of the iterative 
algorithm will not converge to a single point but rather a limit region as illustrated in Fig. [3] We shall formally 
define region stability below. 

Definition 3 (Region Stability of Switched Systems): A discrete-time randomly switched system with state vector 
p(n) is said to be stable w.r.t. a limit region C, if for every trajectory p (re, p(0)), there exists a point of time 
iVo (p(0)) such that from then on, the trajectory is always in the limit region C. Mathematically, 

V p (re, p(0)) , 3 N (p(0)) such that p (n, p(0)) G £, V re > iV (p(0)) . (23) 



Before proceeding further, we introduce the following intermediate definitions. 
Definition 4 (Matrix-2 Norm): The matrix-2 norm ||A|| 2 is defined to be |27l 



max 

IMI 



|Ax| 



2 • 



(24) 



{x: ||x|| 2 = l} 

where the vector norm ||x|| 2 is defined as |27l : ||x|| 2 = Vx T x. 

Definition 5 (Vector Block-maximum Norm): The vector block-maximum norm on the power profile p(n) is 
defined to be E7I 



|p(")llbiock — m a x|| Pfc (r*)|| 2 



(25) 



For ease of elaboration, we also introduce the following contraction modulus 0, E71 : 

{3 k (n k (n))±\\l NF + n- 1 (n)dl k C k (n)\\ 2 + £ HD^V^AMIa ,V fc G AC, (26) 

where c?LCfc(n) G £,N F xN F d eno t es fog second order partial derivative of C k (n) w.r.t. Pj(n) evaluated at p(n), 
i.e., 



9 2 C fc (n 



^w-SS^w-^U^^-^" 5 



,(!)/ 



J 2 )/ 



AN*), 



T 



,V k,j etc, 



(27) 



where ^(,) = -^|P,V S 6 5. 

We now summarize the region stability property of the proposed DSGPA in the following theorem. 

Theorem 1 (Region Stability of DSGPA): Under the conditions that {/?& (D k (n)) <l,V/cG/C,Vra> l}, for 
sufficiently large n, the probability that the iterates p(n) generated by (fT9l being outside the limit region C, can 
be upper bounded by 



PRegion - _ lim _Pr{p(ra) ^ £} < min <{ 1, 



n— >-+oo 







■1. 



(1-P)N) 

where (3 = maxr v k eK,\/ n>i\ {fik (Dfc(n))} is the maximum contraction modulus; iV = - — \ 
sojourn time of the FSMC {h(n)}; the limit region £ is given by 

Q 



£ = Ua = U{pI 



p-p 



('/) 



<<5 



»(«) 



(28) 
is the average 

(29) 

L hlock J 

q=l 

where 5 = max{v g ,reS,g^r} ||P W ~~ P^ 1 1 block denotes the maximum distance between two NEs in {p^ 9 \ V g G Q}, 
with p( g ) denoting the NE of channel state g. The limit region is the convex hull lf29l of the NEs {p^,V q G Q) 
corresponding to different channel states. As a result, the limit region is a polyhedron with diameter 5 « T T°' , 
where r m ax = max^. "k >max . 

Proof: Please refer to Appendix |A] for the proof. ■ 

Remark 5: In TheoremUl N can be thought as an indicator of the channel fading rate. The larger the N is, i.e., 
the channel changes more slowly, the smaller the PR eg ion is. In particular, we have Pu eg i on = O (l/-ZV) 

D. Asymptotic Order of Growth of the Tracking Errors 

Steady state tracking error is the main concern when designing an iterative algorithm in a time-varying environ- 
ment. Here, we consider the expected-absolute-error (EAE) and the mean-square-error (MSE), which are formally 
defined as follows. 

Definition 6 (EAE and MSE): The EAE (or MSE) is defined to be the expectation of the distance (or squared 
distance) between the iterate p(n) and the corresponding NE pW, i.e., 



EAE (p(n)) = E {h(n)} { 



pin) 



P 



(?) 



MSE(p(n))=E {h(n)} <^ p(n)-p 



■M 



block 
2 

block 



}• 



(30) 
(31) 



EAE (p(n)) = O ( - H ns _ ) , and MSE (p(n)) = O ( V —^ -?—£- , (32) 



where the expectation shall be taken over the stationary distribution of the FMSC {h(n)}. ■ 

The asymptotic order of growth of the expected-absolute-error EAE (p(n)) and the mean-square-error MSE (p(n)) 

are summarized in the following theorem. 

Theorem 2 (Asymptotic Order of Growth of EAE and MSE): Under the conditions that |/3fc (D&(n)) < 1, V k G 

/C, V n > l}, the order of growth of the expected-absolute-error EAE(p(?i)) and mean-square-error MSE(p(n)) 

during steady state are given by: 

8 \ ( f3 2 (2(3 + (l - B)N) 

!—-= , and MSE p n = O V - V ^—> 

(1-W \(l-f3i)(l-[3)N 2 

where 8 = max{ V fceAC.V n>i} {Pk (Dfc(n))} is the maximum contraction modulus. 

Proof: Please refer to Appendix |B] for the proof. ■ 

Remark 6: The expressions of the tracking errors EAE and MSE given in Theorem \2\ depend on the average 

sojourn time N, which can be thought as an indicator of the channel fading rate. The larger the N is, i.e., the more 

slowly the channel changes, the smaller are the tracking errors. Particularly, we have EAE (p(n)) = O (l/-/V) and 

MSE(p(n)) = 0(l/N). 

IV. Tracking Error Optimization 

In the previous section, we have established the region stability property of the proposed DSGPA and derived the 
order of growth of the tracking errors. In this section, we shall design the scaling matrices {Dj.(n), V k G /C, V n > 1} 
to minimize the tracking errors. Specifically, we shall first construct a dominated error process. Based on that, we 
shall optimize the scaling matrices via the Markov decision process (MDP) modeling and show that the optimal 
scaling matrixes can be computed distributively. 

A. Tracking Error Optimization 

In this section, we shall derive an tracking error optimal scaling matrices for the proposed DSGPA. For ease of 
elaboration, we first introduce the the notion of stage below. 

Definition 7 (Stage): A Stage is defined as the time-span, for which the channel fading process {h(t)} remains 
unchanged (i.e., stays at the same channel state), as illustrated in Fig. [4] ■ 

We next proceed to construct a dominated error process {e(m)} defined as: 

e(m + 1) = e(m)(t>m m + $m,m+i, V m = 1, 2, • • • , (33) 

where 4> m denotes the worse case contraction modulus of all the transmitters at the m th stage; 5 m . m+ i = ||p(m) — p(m + 1) || bl0( 
denotes the distance between the NE p(m) of the m th stage and the NE p(m + 1) of the (m + l) th stage. The 
dominating error process {e(m)} has the following properties. 

Lemma 1 (Property of Dominating Error Process): The dominating error process {e(m)} defined in equation 
(|33l is an upper bound of the distance (w.r.t. the vector block-maximum norm) between the algorithm trajectory 
{p(n)} and the NE at the beginning of the m th stage at the steady state, i.e. 

. . „ . 5f3 
< e(m) < e(m) < -, almost surely (a.s.), V m = 1, 2, • • • , (34) 

where {e(m)} denotes the actual initial error of each stage, and we choose e(l) = e(l). 

Proof: Please refer to Appendix O for the proof. ■ 

Let x( n ) = (e(n),h(n)) € M+ (£)C K NfX1 denote the system state at the n th time-slot, where it is assumed 
that e(n) = e(m) when the n th time-slot is in the span of the m th stage. For a given system state realization 
x(n), the transmitters adjust the scaling matrix action D(n) according to a stationary scaling matrix control policy 
7r = D (x(n)) defined below. 

Definition 8 (Stationary Scaling Matrix Control Policy): A stationary scaling matrix control policy it : R + C k ' Nf x l — > 
(^kn f xKN f j s d e fi nec i as th e mapping from the currently observed system state x(n) to a scaling matrix action 
D(n). ■ 
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Using Lemma [7J we shall derive an optimal scaling matrix control policy w.r.t. an average tracking error upper 
bound (represented by e(n)). We further assume that the dominating error process {e(n)} admits finite values 8 in 
£ = {ei, e-2, • • • , c-l)- As a result, given a stationary scaling matrix control policy it, {x( n )} i s an induced Markov 
Chain, and the transition probability of x( n ) i s given by: 

Pr{ X (n + l) = (e h h r )\ X (n) = (e 4 ,h g ),7r ( X (n))} = (35) 

/ - r \ K 2 N F log,,, , ,, v 

T qr (l - *«>»') (^l) " X<)) ,Vl<M<L,l<,,r<Q, 

where T qr = [T] is the transition probability from state q to state r of the FSMC {h(n)}; 5 qr = ||p^ — p^ || block 
is the distance between the NE p( 9 ) of the m th stage and the NE p( r ) of the (m + l) th stage; and j3 (x( n )) = 
max^jt {/3fc (Dfe(n))} is the maximum contraction modulus at time-slot n. 

The tracking error optimization problem is formally given below. 

Problem 2 (Tracking Error Control Problem): To minimize the average tracking error, the adaptive scaling ma- 
trix control policy ir* is given by 

1 - 
vr* = argmin J n , with J n = lim sup -VE T { 5 (e(n))} , (36) 

where g (e(n)) is an increasing function of {e(n)}, which measures the per-stage tracking error and E 71 " denotes 
the expectation w.r.t. the induced measure (induced by the control policy n). 

In general, the optimization problem in (l36l ) is very difficult to solve due to the huge dimensions of variables 
(control policy) involved as well as difficulty to express the optimization objective function E 71 " {g (e(n))} explicitly 
as the variable ir. Yet, utilizing the special structure of the transition kernel in (1331 ), the solution of Problem \2\ is 
summarized in the following theorem, which shows that the adaptive scaling matrices {Dfc(ra),V/c G JC\ can be 
computed independently in every time-slot. 

Theorem 3 (Solution of the Tracking Error Control Problem): The optimal scaling matrix control policy for Prob- 
lem |2] is given by the solution of the following optimization problem. 

minimize maXfc6/c/3fc ( Dfc (n)) 

subject to max fce/c /3fc (D fc (n)) < 1, (37) 

D(n) >- 



where /3k (Dfe(n)) denotes the contraction modulus, which is defined in equation (I26I ). 

Proof: Please refer to Appendix O for the proof. ■ 

B. Distributed Implementation of the Optimal Scaling Matrices 

By exploiting the block-diagonal structure of D(n), the optimization problem (l37l) can be naturally decoupled 
into K subproblems and solved distributively at the K transmitters. The subproblem that needs to be solved at the 
fjh t ransm jtter can be formulated as follows. 

Problem 3 (Subproblem of Scaling Matrix Optimization): The optimal scaling matrix Dj.(n) at the k th transmit- 
ter for minimizing the tracking errors is given by the solution of the following problem, V k G /C. 

mi S ze a( d *(-)) 

subject to /3fc(D fc (n)) < 1, (38) 

D fe (n) y 0. 

Since the objective function j3k (Dfe(n)) of Problem \3\ consists of linear functions of the positive definite matrix 
Dfc(n) and sum of matrix norms, Problem\3\is a convex optimization problem ||29l . However, the objective function 
Pk (Dfc(n)) is not differentiable. To find a closed-form solution of Problem\3\ we need the following intermediate 
results. 

8 Since {e(n)} is bounded, we can always set a realization of {e(n)} to a larger nearest integer, which results in a finite integer-valued 
random process {e(n)}. 



II 

Lemma 2 (Objective Function of Problem^: The objective function j3 k (D k (n)) of Problem \3\ can be rewritten 

as 

K 

P k {T> k {n)) = \\l NF +T>-\n)dl k C k (n)\\ 2 + ]T ||D^(n)a^C fc (n)|| 2 , V k G /C, (39) 

and can be lower bounded by 

/3 fc (D fe (n)) > V maxfr ,V fc G /C,V p(n) G 0. (40) 

Proof: Please refer to Appendix |E] for the proof. ■ 

By virtue of Lemma\2\ we can get a closed-form solution for Problem\3\ We summarize the main results of this 

section into the following theorem. 

Theorem 4 (Optimal Solution of the Tracking Error Control Problem): The optimal solution ~D k (n) of Problem 

13 is given by 

B k (n) = -8% k C k {n),V k e K. (41) 

Proof: Please refer to Appendix |F] for the proof. ■ 

From Theorem |?] we know that the optimal scaling matrix D^ 1 (n) is given by the minus inverse of the second 
order partial derivative of the capacity function C k (n) w.r.t. the power allocation vector p k (n), V k G /C. Therefore, 
the scaling matrix Dr (n) can be computed at the k th transmitter based on local information only. Theorem\4\also 
implies that the smallest achievable value of f3 k (D k (n)) is the lower bound given by (l40l) . Based on that, we get 
an alternative sufficient condition for the iteration ( fT9b to be a block-contraction mapping w.r.t. the block-maximum 
norm for each subsystem q G Q, which is summarized into the following corollary. 

Corollary 1 (An Alternative Sufficient Condition): When the scaling matrices are chosen to be T) k (n) = —d kk C k (n) 
/C, an alternative sufficient condition for iteration ( fT9l to be a block-contraction mapping w.r.t. the vector block- 
maximum norm is given by 



> max —r^- 
, *eS n (s) fo) 



q^(q) 
max I V max -^ J- < 1, V g G Q, (42) 



,W/„\ A », 



where g k Hq) = g k j (n) denotes the power gain when the FSMC {h(n)} is in state q € Q. 

Remark 7: The condition given in equation (l42l coincides with the condition given in Theorem 3 of |3T1 with the 
weight vector chosen to be 1. Note that the condition given in PT1 is for the iterative water-filling algorithm (IWFA) 
|T3l , while Corollary |7] here is concerned with the proposed DSGPA. While the conclusion that the simultaneous 
DSGPA has similar convergence speed as the simultaneous IWFA given in Q is based on numerical experiments, 
here we establish a theoretical foundation for that conclusion. 



V. Numerical Results and Discussions 

In this section, we shall compare the proposed DSGPA with three baseline schemes: (I) Baseline 1: gradient 
projection algorithm with a general positive definite scaling matrix (Gen-GPA); (II) Baseline 2: gradient projection 
algorithm with a diagonal scaling matrix (Dia-GPA), whose diagonal entries are the diagonal entries of the 
corresponding Hessian matrix |9), |[27l - |[29l ; (III) Baseline 3: gradient projection algorithm with a constant stepsize 
(Con-GPA) £ = 0.005 0, [271, [29]], [32]]. We choose these three baselines as they have covered the majority of the 
existing approaches (baseline 2 and baseline 3) and also have an idea about what's the best possible performance 
(baseline 1). Specifically, baseline 1 does not impose any block diagonal structure on the scaling matrices and 
requires centralized implementation. Hence, comparison with this baseline illustrates the potential performance loss 
of our proposed scheme due to the imposed block-diagonal structure in the scaling matrices. For baseline 2 and 
baseline 3, they are used extensively in O, [271 -1291 and (31, [27l , [29l , [321 , respectively. Comparison with these 
baselines illustrates the performance improvement of the proposed scheme over these existing approaches. 
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In all the simulations, there are 10 randomly placed transmitter-receiver pairs, sharing 32 independent subbands, 
i.e., it is chosen that K = 10, Np = 32. The total bandwidth is 10 MHz. The maximum transmit power at the k th 
transmitter is set to be 1 Watt, i.e., Pk, ma x = l,Vfc G /C. The distance from the k th transmitter to the j th (Vj ^ k) 
receiver is set to be 400 meters, while the distance from the k th transmitter to the k th receiver is set to be 100 
meters. The path-loss exponent is 3.5. The small scale fading channel gain is generated according to the distribution 
CA/"(0; 1). Moreover, we choose qJJ* = 4 for the FSMC {4? (")}' V k,j EtC,seS and the state space H^) of 

the FSMC < h^Hn) > is constructed based on the receiving SNR partition approach |[22l . |23l . 

A. Tracking Performance Comparison 

Fig. [5] illustrates the normalized sum-utility versus time-slot index for the proposed DSGPA and the three baseline 
schemes. As illustrated, the proposed DSGPA has a much better tracking capability than the baseline schemes Con- 
GPA and Dia-GPA, which are designed for quasi-static CSI. On the other hand, the DSGPA has similar performance 
as the centralized solution Gen-GPA, which shows that performance loss incurred by the block-diagonal structure 
of the scaling matrix used in the proposed DSGPA is negligible. 

B. Region Stability Property 

Fig. [6] shows the simulation results of region stability. The simulation results are consistent with the analytical 
results stated in Theorem^ i.e., the probability that the algorithm trajectory at steady state being out of the limit 
region C (see equation (|29l l) is proportional to the normalized update interval 1/N. Moreover, as the scaling matrices 
in the proposed DSGPA are adaptive to the time-varying CSI, the DSGPA performs better than the baseline schemes 
Con-GPA and Dia-GPA . On the other hand, the performance of the DSGPA and the centralized solution Gen-GPA 
are similar. 

C. Order of Growth of the Tracking Errors 

Fig. [7] and Fig. [8] show the simulation results of the expected-absolute-error (EAE) and the mean-square-error 
(MSE), respectively. Both figures are consistent with the analytical results given in Theorem\2\ i.e., the tracking 
errors, namely EAE and MSE, are proportional to the normalized update interval 1/N. Moreover, as the scaling 
matrices in the proposed DSGPA are adaptive to the time-varying CSI, the tracking errors associated with the 
DSGPA are much smaller than the baseline schemes Con-GPA and Dia-GPA . On the other hand, the performance 
of the DSGPA and the centralized solution Gen-GPA are quite similar. 

VI. Conclusions 

In this paper, we have proposed a distributed scaled gradient projection algorithm (DSGPA) to solve the power 
control game in a K pair multicarrier interference network under the finite-state Markov channel (FSMC) model. 
We have shown that the proposed DSGPA converges to a limit region rather than a single point under the FSMC 
model. We have also shown that the order of growth of the tracking errors, namely the expected-absolute-error 
(EAE) and the mean-square-error (MSE), is given by O (l/iV). By exploiting the Markovian property of the FSMC, 
the scaling matrix optimization problem (w.r.t. tracking error) is modeled as an infinite horizon average cost MDP 
While there is no simple solution for MDP problems, we exploit the specific structure in the transition kernel and 
derive a low complexity distributive solution for controlling the scaling matrices to minimize the tracking errors. 
Simulations are done to verify the analytical results as well as to demonstrate the superior performance of the 
proposed DSGPA over three baseline schemes. 



Appendix A 
PROOF OF Theorem [7] 

Consider a time interval [0, N] with M switchings, i.e., there are M stages in the interval [0, N], Let {q\ , q2, ■ ■ ■ , <?m}> 
{Ni, N2, • • • , Nm} and {(pi, <fa, • • • , 4>m} denote the channel states, sojourn times and contraction modulus of the 
M stages, respectively, as shown in Fig. 01 Under the conditions that {/% (D^(n)) <l,V£;€/C,Vre> l}, the 
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iteration ( fT9l ) is block-contraction in each stage (3j, |[27l . As a result, the distances {||e(m)|| block , 1 <m< M} 
between the iterate p(n) and the NE at the end of each stage can be upper bounded by 

l|e(l)|| block <p(0)<\ (43) 

l|eM|| block < ||e(m- l)|| block ^ m + 5 m - 1>m , 2<m<M, (44) 

where b m -\^ m denotes the distance between the NE of the (m — l) th stage and m th stage, i.e., the jump of the 
equilibrium point of the switched system. Iterating equation (l44l from m = 1 to m = M, we get 

M M M 

j,N m 



ie(M)ii block < P (o)n^+En*» 

m=l i=2 m=i 

M M 

<p(o))9^^-+*x;ii^" < 45 > 



Z=2 m=Z 



where /3 = max m ^ m = max{ fc „} /?& (Dfc(n)) is the worst case contraction modulus. 

Let q = [<7i <72 " " ' Qm] an d N = [N\ N 2 • • • Nm], then take expectation of both sides of equation (1431 ) w.r.t. 
{q, N}, we can get 



M M 



E {q , N} {||e(M)|| block } < E {q , N} {p(0)/3^=^} +E {q , N} ^ II ^ • < 46 ) 

Under the assumption that [T] = i/ K Np ,V q € Q (see equation (0])), then we know that N±, N2, ■ ■ ■ , A^m are 
identically distributed with probability mass function (PMF) given by 

Pr {N m = l} = v K2N ^-V (l - „*»**■) , V I = 1, 2, • • • . (47) 

Then, we have 

MM ~\ ( (MM "Mc/i JW-1\ 



e rn } ^e n ? Nm = e m e {n> ^e n p 1 



\ 1=2 m=l J \ \ 1=2 m=l 

where a = E {/3 N ™} = f^^f = p + ({_p )7r Therefore, let M -> +00, we get 

ML. .T<^ 

a (1-P)N 



fel-a J 

q ^ = — ^ '-, (48) 



1 — a 



E {q . N1 {||e(oo)| Lc J<^ = -^-=. (49) 



P'-{l|eMII M „ t ><S}< - „,^ , (50) 



Then, by virtue of the Markov inequality we get 

A 

(1-(3)N 
which means 

lim Pr{p(n)^£}<mm{l ^ 1 . (51) 

n^+00 [ (l-[3)N) 

The the sojourn time of the FSMC {h(n)} is geometrically distributed with parameter v K Nf , and the average 
sojourn time of the FSMC {h(n)} is given by - — ^ . 
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Appendix B 
PROOF OF Theorem [2] 

From equation (|49l ) in Appendix\A\we know that, at steady state (i.e., when n — > +00), 

5/3 



EAE(p(n)) < 



(1-/3)N 



(52) 



Letw = E{^-.}-^=^^ 



- ^(few we then have 



E 



' M M 
1=2 m=l 



N„ 



E (q} < E {N} 



2ru 
1 — w 



1 - a 17 " 1 ro A/ - ^c^" 1 




1 — w 



1 — a w — a 

Combining equation (I46t and equation (|53T ). and let M — > +00, we can get 

5 2 [3 2 (2/3 + (1 - /3)iV) 



MSE(p(n)) < 



(l-/32)(l_/3)iV 2 



(53) 



(54) 



Appendix C 
Proof of Lemma Ul 

Under the conditions that {/3fc (Dfc(n)) <l,Vfc£/C,Vn> l}, the iteration (fT9l) is block-contraction in each 
stage 0, IT271 . From e(l) = e(l), we can get 

e(l+)<e(l)/?j Vl =e(l)/3f 1 , (55) 

where e(l+) denotes the distance between the algorithm trajectory and the NE at the end of stage 1. Since 
e(2) < e(l+) + Si t 2, we then have 

e(2) < e(l)/3^ + <y lj2 = e(l)^ + <Ji, 2 = e(2). (56) 

Repeat the same procedure for each stage (i.e., m = 3, 4, • • •), we can get 

e(m) < e(m), V m = 1,2, • • • . (57) 

Moreover, set N m = 1,V m, in equation (|45T ) and let M — >• +00, we can get e(m) < y^j. Since the above 
arguments hold for all realizations of the FSMC (h(n)}, equation d34l ) holds almost surely. 

Appendix D 
Proof OF Theorem \3\ 

Define P (xir\xiq, n (Xiq)) - Pr {x(n + 1) = (ei,h r )\x(n) = (e i; h g ),vr (%(n))}, the optimal control policy 
ir* can be found by solving the Bellman equation below 11331 : 



V (Xiq) + = minimize < g (e t ) + Y] P (;\> |x ig , tt (x ig 



))^(Xir) 



(58) 



where V (xiq) is called the optimal value function at state Xiq'i and Q is the optimal average cost. The Bellman 
equation is a fixed point equation w.r.t. the variables {V (Xiq) , 1 < *, ^? < Q} and 9, which is derived using the 
principle of divide-and-conquer from the original MDP in (|36l ). It is shown using the theory of MDP |f33l that 
the fixed point solution 6 of (1581 ) would give the optimal value of J 71 " in (l36l ) and the optimal control policy tt* is 
given by the solution of the RHS of (|58l ) w.r.t. the fixed point solution {V (xtq) > 1 < h Q < Q}< F° r tne transition 
kernel P (xzr | Xig , vr (x ig )) given in $35$, we have 



— < 1, since e\ < e«[/3(n)] q + 5 gr < €4 + # gr , where 
A^q denotes the sojourn time of state q of the FSMC {h(n)}. As a result, minimizing /3(n) can simultaneously 
minimize all the transition probabilities P (xirlxiq, 71 (Xiq)) , V 1 < / < L, 1 < r < Q, which suggests that the 
optimal control policy tt* is to take the minimum possible /3(n) at state Xiqi V 1<*<^, 1<^< Q- Moreover, 
since /3(n) = maxjy fc e /c} {A (Dfc(ra))}, the minimum possible (3(n) is given by solving the optimization problem 
given in equation (l37l) at each time-slot. 
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Appendix E 
Proof of Lemma E] 

To prove Lemma \2\ we need the the following intermediate results. 

Lemma 3 (Matrix Induced-2 Norm of Product of Two Matrices): For two symmetric positive definite matrices 
A,B g C mxm , we have 

|| AB|| 2 > \rnin (A) ||B|| 2 , (59) 

where A m j n (A) denotes the minimum eigenvalue of matrix A. 

Proof 1: Given that A is positive definite, it then follows that A 2 — X 2 nin (A) I m is positive semidefinite. As B 
is also positive definite, then (A 2 — X 2 min (A) I m ) B 2 is positive semidefinite [34]. Denoting a G C m and b G C m 
as the unit norm eigenvectors corresponding to the largest eigenvalues of matrices A 2 B 2 and B 2 , respectively, we 
have 



IABI 



a tA 2 B 2 a > btA 2 B 2 b > X 2 mm (A) btB 2 b = X 2 mm (A) ||B| 



2 
2 ' 



Therefore, from the definition of matrix-2 norm 1341 . we get || AB|| 2 > X m in (A) ||B|| 2 . 

gff(n) gffto 

-D^7 1 (n)9 2 fc Cfc(n), we then have 



We next proceed to prove Lemma [2] Define F(n) 



-diag 






K 



A (D fc (n)) = 
Moreover, from Lemma \3\ we know that 



LTVj, 



Dfcfn' 



+ J] D fc (n)F(n) 



j=l,j^k 



K 



Pk (Dfc(n)) > 



1- A, 



(a) 



D fc (V 



D fc (n 



». 



+ A min (D fc (n)j Yl P( 

X (6) * 




1 ] > E || F ( n ) 

j=l,j^k 



(60) 
and Dfc(n) = 

(61) 

(62) 
(63) 



where (a) is because A m j n I D& I < 1 achieves smaller fty. (Dfc(rt)) than A m j n ( D^ ) > 1; and (b) is because 
Pk (Dfc(n)) < 1 and choosing X m in \^k) = 1 is the best possible choice. 

Appendix F 
PROOF OF Theorem @ 



Similar to equation (l62l . we can also get 



K 



(3(D k (n))> 



X, 



D fc (n))-1 +A min (D fe (n)J ^ F(n 



j=ij^ 



Dfcfn 



(64) 



1, 



Equation (l62l together with equation (l64l imply that we shall choose X max (Dfc(? 
which achieves the lower bound of /3 (Dj.(n)). Therefore, one of the optimal solutions of Problem\3\is given by 
Dfc(n) = I Nf , i-e., D fc (n) = -dl k C k {n). 
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TABLE I 

Summary of Main Notations 



Notation 


Meaning 


n 


an index variable to denote the time-slot 


m 


an index variable to denote the stage 


h%(n) 


channel gain between the k th receiver and the j th transmitter on the s th subband 


h(n) 


collection of the channel coefficients of the entire network 


D fc (n) 


scaling matrix of the k th transmitter at time-slot n 


D(n) 


block-diagonal matrix, which consists of the K scaling matrices {Dfe(n),Vfc 6 K.} 


/3fe(D fc (n)) 


contraction modulus of the k th transmitter at time-slot n 


P 


worse case contraction modulus of all transmitters in all channel states (i.e., all time-slots) 


4>m 


worse case contraction modulus of all transmitters in the m th stage 


N q 


random variable, which is the sojourn time of channel state q 


N m 


random variable, which is the sojourn time of stage m 


pW 


NE of Game Q at channel state q 


5 


maximum distance between two NEs corresponding to two different channel states 


£ 


limit region, which is a polyhedron with diameter S 


{e(m)} 


dominated error process 


Om,m+l 


distance between the two NEs corresponding to the m th stage and the (m + l) th stage 


7T 


scaling matrix control policy 


x(«) 


system state at the n th time-slot 


(X(n)} 


Markov chain induced by the control policy n 



( TX 2 ^ ^ 



& 



♦^ RXkJ 



-► Communicating Link 
-► Interfering Link 



Fig. 1. A sample multicarrier interference network with K transmitter-receiver pairs, where the k t% transmitter wishes to communicate 
with the k th receiver, V k G /C. All the K transmitters share Nf nonoverlapping subcarriers. 



n x 2 




• Equilibrium Point 

► Algorithm Trajectory 

State-vector x = [xi X2] T 

Fig. 2. A pictorial view of an algorithm trajectory converging to the static Nash Equilibrium point in a two-dimensional algorithm space. 
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4 x 




Fig. 3. An illustration of the region stability of a randomly switched system with one-dimensional state space 1121 . From time-slot No and 
onwards, the trajectory remains in the limit region with probability Pn eg ion given in equation j28l l. 
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Fig. 4. An illustration of the M stages that the switched system d!9t has gone through. 4> m and N m denote the contraction modulus and 
the sojourn time of the m th stage, respectively; and <5 mi7n +i denotes the distance between the NE of the m th stage and the (m+ 1) stage, 
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Fig. 5. Tracking performance comparison of the proposed DSGPA and the baseline schemes: Gen-GPA, Dia-GPA and Con-GPA. The link 
capacity is normalized to the maximum link capacity obtained across all the time-slots. 
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Fig. 6. Region stability property of the proposed DSGPA and the baseline schemes: Gen-GPA, Dia-GPA and Con-GPA. The asymptotic 
order of growth refers to the results stated in Theorem [7] (see equation d28b). 
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Fig. 7. Expected-absolute-error (EAE) versus Inverse of the Average Sojourn Time of the proposed DSGPA and the baseline schemes: 
Gen-GPA, Dia-GPA and Con-GPA. The EAE is normalized to the maximum EAE of all the schemes. 
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Fig. 8. Mean-square-error (MSE) versus Inverse of the Average Sojourn Time of the proposed DSGPA and the baseline schemes: Gen-GPA, 
Dia-GPA and Con-GPA. The MSE is normalized to the maximum MSE of all the schemes. 



